Method for reducing noise for coding of noisy images or image sequences

ABSTRACT

A prediction error (e q [x,y]) is added to a predicted frame ({circumflex over (f)}[x,y]) or a predicted block for receiving a decoded frame (g q [x,y]) or a decoded block to be further used in a prediction loop by an encoder or to be sent to the output of a decoder. The reference frame (g q [x,y]) or the reference block includes a useful signal part and a noise signal part. The reference frame (g q [x,y]) or reference block pass through a dedicated noise reducing filter to reduce or eliminate the noise signal part of the reference frame (g q [x,y]) or reference block.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is the U.S. national stage of International ApplicationNo. PCT/EP2011/050321 filed Jan. 12, 2011, and claims the benefitthereof. The International Application claims the benefits of EuropeanApplication No. EP10000540 filed on Jan. 20, 2010 and EuropeanApplication No. EP10007801 filed on Jul. 27, 2010, all applications areincorporated by reference herein in their entirety.

BACKGROUND

Described below is a method for reducing noise for coding of noisyimages or image sequences.

Professional video applications such as monitoring systems forbuildings, industrial manufacturing and medical applications need videosignals with very high quality. These signals are likely to have veryhigh resolutions in spatial as well as temporal direction, so theuncompressed data can become very large. It is therefore important tocompress these signals as much as possible without visible informationloss. Recent video compression systems exploit the temporal correlationbetween images, but they are optimized for consumer qualityapplications.

Usually, image sequences that are acquired by a physical process can beconsidered to be degraded by noise caused through the acquisitionprocess. Noise might result from sensor noise, amplifier noise orquantization noise, and so on. It has been found that at very highquality so-called inter-frame coding becomes less efficient relative toso-called intra-frame coding. An inter-frame is a frame in a videocompression stream which is expressed in terms of one or moreneighbouring frames. The “inter” part of the term refers to the use ofinter-frame prediction. This kind of prediction tries to take advantagefrom temporal redundancy between neighbouring frames in order to achievehigher compression rates. The term intra-frame coding refers to the factthat the various lossless and lossy compression techniques are performedrelative to information that is contained only within the current frameand not relative to any other frame in the video sequence. In otherwords, no temporal processing is performed outside of the currentpicture or frame. It is expected that the reason for the differentefficiencies of inter- and intra-frame coding is (additive) noise withina video signal which effects the motion compensated prediction inside avideo encoder/decoder.

In order to improve the coding efficiency, different algorithms can beapplied to the noisy image sequences prior to encoding.

The noise pre-filtering can be done inside the video encoder. Thus, thebitrate for coding a noisy image sequence is reduced while itssubjective quality is improved. However, the filtering processintroduces errors, which is not allowed in lossless compression of videosignals.

Another approach for bitrate reduction is in-loop filtering of lossyencoded video data. In lossy coding a deblocking filter or quantizationnoise removal filter can improve the coding efficiency as well as thevisual quality of the signal. An in-loop denoising filter for reductionof impulse noise leads to a bitrate reduction and subjective qualityimprovement.

SUMMARY

It is therefore desirable to provide a method to improve noise reductionfor coding of noisy image sequences. It is a further desirable toprovide a method to improve noise reduction for lossless coding of noisyimage sequences. Also described below are an encoder and a decoder whichallow an improvement in noise reduction for coding of noisy imagesequences.

The method described below reduces noise in coding of noisy images orimage sequences. The method includes adding a prediction error and apredicted frame or a predicted block for receiving a decoded frame or adecoded block to be further used in a prediction loop by an encoder orto be sent to the output of a decoder, wherein the reference frame orthe reference block includes a respective useful signal part and a noisesignal part. Also, the method includes denoising of the reference frameor reference block only with a dedicated noise reducing filter to reduceor eliminate the noise signal part of the reference frame or referenceblock.

In order to improve the motion compensated prediction for a noisy imageor a noisy image sequence, the method is based on the idea to minimizethe noise within the reference frame (which is an already reconstructedframe) while keeping as much relevant details as possible. As a result,less noise in the reference frame allows a more accurate motionestimation. Furthermore, the difference signal has less energy and thusleads to a bitrate reduction of the coded bitstream. Hence, a betterreference frame for the frame prediction is generated. A furtheradvantage is that a higher compression efficiency can be achieved.

A dedicated noise reducing filter is a filter which is specificallydesigned for filtering noise.

According to an embodiment the reference frame or the reference block ismotion compensated by a motion compensation element to receive thepredicted frame or the predicted block. The motion compensated referenceframe or reference block therefore is the predicted frame or predictedblock, also known as predictor or predictor signal to the person skilledin the art.

According to a further embodiment a noise adaptive filter is used whichis adapted to estimate noise by evaluating the current frame to becoded. By evaluating the current frame the adaptive filter evaluatesnoise nature and statistical properties like mean, variance andautocorrelation function of the noise and so on. Furthermore, the usefulsignal part is analyzed with regard to video/image content, video/imageresolution, statistical properties, and so on. After this evaluation,the noise adaptive filter removes as much noise as possible orappropriate while keeping the relevant details for coding. E.g., darkscenes usually contain more noise than bright image/video content. Theproperties of the noise adaptive filter imply a stronger filtering whenmore noise is present in the reference frame or the reference block thathas to be used for prediction.

Furthermore, a pixel location adaptive filter can be used as noisereducing filter that takes into account the content of the referenceframe or reference block. Especially, the amount of filtering might bedependent on local signal properties. The latter means that edges shouldbe preserved for efficient video compression. For example, differentfilter windows with different sizes for filtering might be used.

In an embodiment, as noise reducing filter a minimum mean squared error(MMSE) estimator (e.g., a Wiener filter as a linear estimator) is used.However, any filter that reduces noise but preserves the image contentcan be used. It is not relevant whether the noise reducing filter haslinear or non-linear properties.

According to a further embodiment, denoising of the reference frame ismade within a prediction loop without directly affecting the predictionerror which is further encoded by the encoder. Denoising within theprediction loop is sometimes referred to as in-loop denoising. Accordingto this suggestion it is possible to make the method applicable to lossyas well as to lossless video coding.

It is further desirable that denoising is made before or after bufferingthe reference frame in a buffer of the prediction loop. Furthermore, thein-loop denoising filter could also be inserted after a motioncompensation element which can be used especially for low noise levels.

According to a further embodiment denoising is applied on each colourchannel of the reference frame separately or on their combination.

For lossless coding, a quantization element which is located in the pathfrom a comparator comparing the current frame and the reference frame toan output outputting the prediction error and frame, respectively, isbypassed.

According to a further embodiment the current block to be coded and thereference block are blocks or regions (i.e., one pixel or an amount ofpixels) of a single frame. Therefore, the block to be coded is called anintra-coded block (i.e., an intra-block).

According to a further embodiment, the same noise reducing filter isarranged in the encoder and the decoder to denoise the reference frameonly. In other words, the same noise reducing (removal) filter has to beapplied in the decoder. Denoising filter parameters of the noisereducing filter of the encoder might be part of the coded bitstream. Incase that the denoising filter parameters are transmitted from theencoder to the decoder, a denoising filter parameter flag might besignalled. If the filter parameter flag is not set the decoder mightestimate the filter parameters (or parts of the filter parameters) byitself. E.g., the noise strength can be estimated similarly at theencoder and the decoder.

According to a further embodiment, in the encoder a current frame to becoded is compared with the predicted frame or a predicted block forreceiving the prediction error to be further used in the prediction loopby the encoder wherein the predicted frame or predicted block includes auseful signal part only (in the case of ideal noise filtering).

According to a further embodiment, denoising is adaptive with respect toa quantization parameter of the quantization element such that denoisingis the lower the higher the quantization parameter (i.e., the higher thequantization step size) is, and vice versa. The denoising algorithm canbe adaptive with respect to the quantization parameter as quantizationof the noisy reference frame could partly have noise reducingcapabilities on the noise part of the reference frame.

Also described is an encoder that adds a prediction error (e_(q)[x, y])and a predicted frame ({circumflex over (f)}[x, y]) or a predicted blockfor receiving a decoded frame (g_(q)[x, y]) or a decoded block to befurther used in a prediction loop by the encoder (ENC) or to be sent tothe output of a decoder (DEC), wherein the reference frame (g_(q)[x, y])or the reference block (FR) include a respective useful signal part anda noise signal part. The encoder further passes the reference frame(g_(q)[x, y]) or reference block (FR) through a dedicated noise reducingfilter (WF) to reduce or eliminate the noise signal part of thereference frame (g_(q)[x, y]) or reference block (FR).

In addition, a decoder is described that adds a prediction error(e_(q)[x, y]) and a predicted frame ({circumflex over (f)}[x, y]) or apredicted block for receiving a decoded frame (g_(q)[x, y]) or a decodedblock to be further used in a prediction loop by an encoder (ENC) or tobe sent to the output of the decoder (DEC), wherein the reference frame(g_(q)[x, y]) or the reference block (FR) a respective useful signalpart and a noise signal part. The decoder additionally passes thereference frame (g_(q)[x, y]) or reference block (FR) through adedicated noise reducing filter (WF) to reduce or eliminate the noisesignal part of the reference frame (g_(q)[x, y]) or reference block(FR).

It is to be noted that the method as well as the encoder and the decodermay be realized in software and may be executed by a computer or aprocessing unit.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects and advantages will become more apparent andmore readily appreciated from the following description of the exemplaryembodiments, taken in conjunction with the accompanying drawings ofwhich.

FIG. 1 is block diagram of a first embodiment of an encoder according tothe method.

FIG. 2 is block diagram of a first embodiment of a decoder according tothe method.

FIG. 3 is an illustration about the possibility to use the method forintra-prediction.

FIGS. 4A and 4B are block diagrams of a second embodiment of an encoderand a decoder, respectively.

FIGS. 5A and 5B are block diagrams of a third embodiment of an encoderand a decoder, respectively.

FIGS. 6A and 6B are block diagrams of a fourth embodiment of an encoderand a decoder, respectively.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Reference will now be made in detail to the preferred embodiments,examples of which are illustrated in the accompanying drawings, whereinlike reference numerals refer to like elements throughout.

The method will be described in more detail with the help of blockdiagrams of different encoders and decoders. For simplification, theblock diagrams are observed at one specific point in time, where onlyone current frame g[x, y] has to be coded and another predicted frame{circumflex over (f)}[x, y] is available.

FIG. 1 shows a block diagram of a lossy video encoder ENC according to afirst embodiment. The current frame g[x, y] and the predicted frame{circumflex over (f)}[x, y] are fed to a comparator 10. At the output ofthe comparator 10 a prediction error e[x, y] is fed to a transformationblock 11. The transformation block 11 which is known as T-blockrepresents any further decorrelation of the prediction error e[x, y],i.e. a (nearly) lossless reversible transformation. The output of thetransformation block 11 is connected to a aquantizer 12, which could bebypassed or skipped for a lossless video encoder. The output of thequantizer 12 is connected to an entropy encoder 13 within the videoencoder. The entropy encoder 13 is connected to an output OUT outputtinga coded bitstream CB.

The predicted frame {circumflex over (f)} is provided by the elements ofa prediction loop. The prediction loop includes an inversetransformation block 14 which is adapted to reverse the transformed andoptionally quantized prediction error e[x, y]. As a result, the inversetransformation block 14 feeds a reversed prediction error e_(q)[x, y] toa first input of an adder 15. At a second input of the adder 15 thepredicted signal {circumflex over (f)}[x, y] is fed. At an output of theadder 15 a signal g_(q)[x, y] which is a reconstructed current frame isfed to a noise reducing filter 16 for filtering noise, e.g. additivenoise. The output of the noise reducing filter 16 is connected to areference picture buffer 17. The reference picture buffer 17 isconnected to a motion estimation and motion compensation element 18within the encoder ENC. At the output of the motion estimation andmotion compensation element 18 the (motion-compensated) reference frame{circumflex over (f)}[x, y] is fed to the comparing and adding elements10 and 15. The motion estimation and motion compensation element 18furthermore receives the current frame g[x, y]. The entropy encoder 13receives control information from the motion estimation and motioncompensation element 18 as well as from the noise reducing filter 16.

The subtraction of the motion compensated reference frame (also calledpredicted frame) from the current frame with the comparator 10 in thevideo encoder ENC is described by the following equation:

e[x,y]=g[x,y]−{circumflex over (f)}[x,y]  (1),

where x and y are spatial coordinates, g[x, y] is the signal whichdescribes the current frame which has to be compressed, {circumflex over(f)}[x, y] is the predicted signal which describes the motioncompensated reference frame and e[x, y] is the prediction error whichhas to be transmitted to the decoder shown in FIG. 2 aftertransformation, optional quantization and entropy encoding. Assumingthat the image sequence is degraded by independent additive white noise,the signals can be written as:

g[x,y]=s _(g) [x,y]+n _(g) [x,y]  (2),

{circumflex over (f)}[x,y]=s _({circumflex over (f)}) [x,y]+n_({circumflex over (f)}) [x,y]  (3),

where s_(g) and s_({circumflex over (f)}), respectively, is the usefulpart of the signal and n_(g) and n_({circumflex over (f)}),respectively, is the noise part of the respective signal. From equations(1), (2) and (3) it follows that

$\begin{matrix}{{{e\left\lbrack {x,y} \right\rbrack} = {\underset{s_{e}{\lbrack{x,y}\rbrack}}{\underset{}{{s_{g}\left\lbrack {x,y} \right\rbrack} - {s_{\hat{f}}\left\lbrack {x,y} \right\rbrack}}} + \underset{n_{e}{\lbrack{x,y}\rbrack}}{\underset{}{{n_{g}\left\lbrack {x,y} \right\rbrack} - {n_{\hat{f}}\left\lbrack {x,y} \right\rbrack}}}}},.} & (4)\end{matrix}$

From equation (4) it can be seen that the prediction error has two majorcomponents. The signal components s_(e)[x, y] can be minimized if thecorrelation between the current noise-free frame and the noise-freereference frame increases. However, the noise component n_(e)[x, y]becomes higher because the noise of the current frame in general isindependent from the noise of the reference frame. In case of additivewhite Gaussian noise, the noise variance of the error image becomestwice as high as the noise variance of one of these two images.

Especially in lossless coding applications the noise part of the currentpicture has to be encoded. In order to increase the compressionefficiency, the noise in the reference frame should be minimized withoutintroducing high degradations in the useful part of the signal. Thiswould allow minimizing the noise amount in the prediction error whereaskeeping the coding process lossless.

The filter 16, which is introduced in FIG. 1 as a denoising filter for(additive) noise, is located on the same place in the encoder diagram asthe usually used deblocking filter in lossy coding applications. Beforesaving the reconstructed image to the reference buffer 17 it should beprocessed by the denoising algorithm which will be explained later.However, denoising with the help of the noise reducing filter 16 alsocould be executed after buffering in the reference buffer 17.

The denoising algorithm is adapted to the noise nature present in thevideo, i.e. the current frames g[x, y]. In order to keep the codingprocess lossless, the same procedure should be applied in the decoderDEC. In FIG. 2, the diagram of a lossless decoder DEC is illustrated. Inthis figure, at an input IN of the decoder DEC the coded bitstream CB isfed to an entropy decoder 30. Reference numeral 31 depicts an inversetransformation block which is connected to an adder 32. At a firstinput, which is connected to the inverse transformation block 31 anerror frame e_(q)[x, y] is fed. At a second input of the icomparator 32,which is connected to a motion compensation element 35 predicted frame{circumflex over (f)}[x, y] is fed. The output of the icomparator 32 isconnected to the predictor loop, forming the same noise reducing filter33 which is used in the encoder ENC, a reference buffer 34 and themotion compensation element 35.

In both the encoder ENC and the decoder DEC the noise reducing elements16 and 33, respectively, are not connected to their outputs OUT.Therefore, the output signal (coded bitstream BC and encoded bitstreamEB) is not influenced by the noise reducing filter.

In the decoder DEC the reference picture is first denoised before it isstored in the reference picture buffer 34. The major difference to aquantization noise removal filter or deblocking filter is that the imagewill be displayed before the noise filtering is applied. Otherwise theencoding and decoding process could not be lossless.

It is desirable to use an adaptive minimum mean squared error estimator(e.g., an adaptive Wiener filter as a linear estimator) algorithm forreduction of the (additive) noise in the reference frame. The filterminimizes the squared error between the noisy and the noise-free imagesequence, which is the objective of a Wiener filter for Gaussianprocesses. Furthermore, the filter prevents smoothing of contours byusing of local statistics of the image signal, which is important foraccurate motion compensation.

If a one of a set of different filters (linear or non-linear) is used asnoise reducing filter 16 at the encoder ENC the choice for the denoisingfilter and its parameters have to be signalled to the decoder to use thesame information. Non-linear filters usually are used in medical imagingapplications or if the Gaussian assumption for the characteristics ofthe useful signal or the noise signal is not correct.

Below, the denoising algorithm will be described in more detail,wherein, as an example, a Wiener filter is used as noise reducingfilter:

In a local region where the signal and the additive noise areuncorrelated and are considered to be stationary, the Wiener filter isdescribed by

$\begin{matrix}{{{H\left( {^{{j\Omega}_{x}},^{{j\Omega}_{y}}} \right)} = \frac{\Phi_{ff}\left( {^{{j\Omega}_{x}},^{{j\Omega}_{y}}} \right)}{{\Phi_{ff}\left( {^{{j\Omega}_{x}},^{{j\Omega}_{y}}} \right)} + {\Phi_{nn}\left( {^{{j\Omega}_{x}},^{{j\Omega}_{y}}} \right)}}},} & (5)\end{matrix}$

where Φ_(ff) (e^(jΩ) ^(x) ,e^(jΩ) ^(y) ) and Φ_(nn) (e^(jΩ) ^(x) ,e^(jΩ)^(y) ) are the power spectral densities of the signal f[x, y] and thenoise n[x, y] respectively. In the case of white noise the powerspectral density is reduced to

Φ_(nn)(e ^(jΩ) ^(x) ,e ^(jΩ) ^(y) )=σ_(n) ²  (6),

where σ_(n) ² is the variance of the noise signal n[x, y], which isassumed to be the space invariant. The signal f[x, y] can be modelled bya sum of a space local mean μ_(f)[x, y] and a space variant localvariance σ_(f) ²[x, y]. If the signal f[x, y] is zero mean the powerspectral density within the local region is reduced to

Φ_(ff)(e ^(jΩ) ^(x) ,e ^(jΩ) ^(y) )=σ_(f) ² [x,y]  (7),

where σ_(f) ²[x, y] is the local variance of the image signal f[x, y].Thus the Wiener filter within the local region is described by

$\begin{matrix}{{H\left( {^{{j\Omega}_{x}},^{{j\Omega}_{y}}} \right)} = {\frac{\sigma_{f}^{2}\left\lbrack {x,y} \right\rbrack}{{\sigma_{f}^{2}\left\lbrack {x,y} \right\rbrack} + \sigma_{n}^{2}}.}} & (8)\end{matrix}$

Generally, the signal f[x, y] is not zero mean and thus the meanμ_(f)[x, y] has to be subtracted from f[x, y] before filtering and hasto be added again after filtering. The filtering process is described bythe following equation:

$\begin{matrix}{{p\left\lbrack {x,y} \right\rbrack} = {{\mu_{f}\left\lbrack {x,y} \right\rbrack} + {\frac{\sigma_{f}^{2}\left\lbrack {x,y} \right\rbrack}{{\sigma_{f}^{2}\left\lbrack {x,y} \right\rbrack} + \sigma_{n}^{2}} \cdot {\left( {{g\left\lbrack {x,y} \right\rbrack} - {\mu_{f}\left\lbrack {x,y} \right\rbrack}} \right).}}}} & (9)\end{matrix}$

The noise variance σ_(n) ² is assumed to be known (i.e., from theacquisition process). As the noise is considered to be zero mean,μ_(f)[x, y] should be equal to μ_(g)[x, y] and the estimation of thelocal mean is reduced to:

$\begin{matrix}{{{{\hat{\mu}}_{f}\left\lbrack {x,y} \right\rbrack} = {\frac{1}{\left( {{2M} + 1} \right)^{2}}{\sum\limits_{k = {x - M}}^{x + M}{\sum\limits_{l = {y - M}}^{y + M}{g\left\lbrack {k,l} \right\rbrack}}}}},} & (10)\end{matrix}$

where M describes the window size in which the image signal isconsidered to be stationary. The estimation of the local variance of thenoise-free image is described by the following equation:

$\begin{matrix}{{{\hat{\sigma}}_{f}^{2}\left\lbrack {x,y} \right\rbrack} = \left\{ \begin{matrix}{{{{\hat{\sigma}}_{g}^{2}\left\lbrack {x,y} \right\rbrack} - \sigma_{n}^{2}},} & {{{if}\mspace{14mu} {{\hat{\sigma}}_{g}^{2}\left\lbrack {x,y} \right\rbrack}} > \sigma_{n}^{2}} \\{0,} & {{else},}\end{matrix} \right.} & (11)\end{matrix}$

where {circumflex over (σ)}_(g) ²[x, y] is the local variance of thedegraded image g[x, y]. The estimation of {circumflex over (σ)}_(g) ²[x,y] is described by the following equation:

$\begin{matrix}{{{\hat{\sigma}}_{g}^{2}\left\lbrack {x,y} \right\rbrack} = {\frac{1}{\left( {{2M} + 1} \right)^{2}}{\sum\limits_{k = {x - M}}^{x + M}{\sum\limits_{l = {y - M}}^{y + M}{\left( {{g\left\lbrack {k,l} \right\rbrack} - {{\hat{\mu}}_{f}\left\lbrack {k,l} \right\rbrack}} \right)^{2}.}}}}} & (12)\end{matrix}$

From these equations it is clear that noise reduction is obtained byaveraging the pixel values in a rectangular window in dependency on thelocal signal and noise variances. Choosing a larger window by variationof M may lead to higher noise reduction, but it may also introduce moreblurring of the useful signal part. Therefore, M can be considered as anoptimization parameter for the compression. Also the variation of σ_(n)² can be considered as an optimization parameter. Increasing σ_(n) ²leads to higher noise reduction whereas the useful signal part is moreblurred.

As already stated above, the major property of the noise reducing filter16 of the encoder ENC and 33 of the decoder DEC is their adaption tonoise, i.e. the nature of the noise and statistical noise propertieslike mean, variance, autocorrelation function, and its adoption to theimage signal, i.e. video/image content, video/image resolution andstatistical properties. The filter in the encoder and decoder remove asmuch noise as possible or appropriate while keeping relevant details forcoding and decoding. This implies a stronger filtering when more noiseis present in the input data that have to be coded. On the other hand,the amount of noise filtering is dependent on the local signalproperties. This means, for example edges will be preserved forefficient video compression. The filtering not only is noise-adaptivebut also pixel location adaptive, e.g. by using different windows withdifferent size for filtering.

The in-loop denoising as described above can be applied on each colourchannel separately. It is also possible to apply the in-loop denoisingon the combination of all colour channels.

It is to be noted that also in the encoder and decoder diagrams of FIGS.1 and 2 the in-loop denoising filter 16 and 33, respectively, can beinserted after reference frame buffer 17 and 34, respectively, withoutany effect on the algorithm. In a further alternative, the in-loopdenoising filter 16 and 33, respectively, could be inserted after motionestimation and motion control element 18 and motion control element 35,respectively. This alternative could be used especially for low noiselevels.

As already noted, for lossless coding the quantization element 12 mightby bypassed or skipped. By doing this the prediction error e_(q)[x, y]becomes e[x, y] and g_(q)[x, y] becomes g[x, y].

Within the encoder ENC a calculation of sub-pixels may be made with themotion estimation and motion compensation element 18. The algorithm maybe enhanced by an interpolation denoising filter. The interpolationdenoising filter is derived from the noise reducing filter 16. It coulduse similar properties and calculates adaptively the interpolationfilter coefficients. As a result, the interpolation is noise-adaptiveand pixel location adaptive with regard to edges and edge preserving.

The denoising filter parameters (filter properties, knowledge aboutnoise and knowledge about the signal) are known or are estimated at theencoder and are transmitted to the decoder as a side information.Alternatively, some or all parameters may be derived at the decoderside. The transmission of the filter parameter may be signalled by adenoising filter parameter flag. If the filter parameter flag is notset, the decoder estimates the filter parameter by itself.

Furthermore, the denoising algorithm may be adaptive with respect to aquantization parameter of the quantization element 12 of the encoder inFIG. 1 as quantization noise masks the noise within the input signal.The bigger the quantization parameter is the less will be filtered withthe noise reduction filter, and vice versa.

In general, noise estimator and its parameters can be selected at theencoder side. This choice can be signalled to the decoder. This meansthe information which algorithm will be used for an estimation of noisecan be selected at the decoder side. The same algorithm should be usedat the decoder side, as already noted.

The procedure of the method has been explained by reference tointer-frame coding where the current frame and the reference frame areframes of different images of an image sequence. However, for coding ofnoisy image sequences or single images the proposed method can also beapplied to intra-coding. This is illustrated in FIG. 3.

In FIG. 3 an amount of blocks FR (i.e., regions of a frame, i.e., anamount of pixels) of a single image is illustrated. Each of the blocksFR has already been encoded. A block depicted with FR′ is to be encoded.Each of the blocks FR includes a number of already decoded samples S.Samples which have to be predicted within block FR′ are denoted with S′.In order to increase the compression efficiency, after encoding samplesS, they are denoised by a causal denoising filter which has similarproperties as the above mentioned in-loop denoising filter. It is to benoted that it is not relevant which intra-prediction method is chosen.E.g., directional prediction from neighbouring pixels of the block to bepredicted (FR′) can be applied. In this case, the reference block to bedenoised is an artificially generated prediction signal forintra-coding.

FIGS. 4 to 6 show further embodiments of a decoder and encoder,respectively, executing the method described above. The alreadydescribed in-loop filtering with respect to different applications, i.e.noise removal, quantization noise removal, deblocking and visual qualityimprovement, is separated into different major independent filters.

In FIG. 4A the in-loop filter is formed of a compression filter 19 and avisual filter 20. The compression filter 19 is responsible forgeneration of the best possible predictor in order to reduce the bitrateand is therefore at a position as the noise reducing filter 16 in FIG. 1(so-called in-loop filter). The compression filter 19 is not optimizedaccording to the visual quality of the output signal. The visual filter20 which is an out-of-loop filter is responsible for the subjectiveand/or objective quality of the output signal. Therefore, the visualfilter 20 is connected to the adder 15 with its input and to the entropyencoder 13 with its output. The two filters 19 and 20 may be optimizedindependently of each other.

At the decoder side illustrated in FIG. 4B the compression filter 36 isthe in-loop-filter part at a position of the noise reducing filter 33 inFIG. 2. The visual filter 37 is connected to the output of the decoderand is an out-of-loop filter.

In the third embodiment of FIG. 5A illustrating the encoder and FIG. 5Billustrating the decoder the visual filter also can be part of theprediction loop.

FIG. 6A and FIG. 6B show a combination of the second and thirdembodiment, where a visual filter 20 a and 37 a, respectively, isarranged within the prediction loop and a second visual filter 20 b and37 b is located outside the prediction loop. As in the second embodimentin FIGS. 4A and 4B the compression filters 19 and 36, respectively, arepositioned within the prediction loop of the encoder and decoder,respectively. This embodiment is of advantage in case that visualfiltering may improve in-loop filtering for prediction. In this case adouble filtering can be avoided. Therefore, complexity of the encoderand decoder can be reduced.

A description has been provided with particular reference to preferredembodiments thereof and examples, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the claims which may include the phrase “at least one of A, B and C”as an alternative expression that means one or more of A, B and C may beused, contrary to the holding in Superguide v. DIRECTV, 358 F3d 870, 69USPQ2d 1865 (Fed. Cir. 2004).

1-16. (canceled)
 17. A method for reducing noise for coding of noisyimages or image sequences, comprising: adding a prediction error, usedby an encoder to produce one of an encoded frame and an encoded block,to one of a reference frame and a reference block that includes a usefulsignal part and a noise signal part; filtering the one of the referenceframe and the reference block only with a dedicated noise reducingfilter to one of reduce and eliminate the noise signal part of the oneof the reference frame and the reference block; and generating theprediction error from one of a current frame and a current block, and atleast part of a current or previous one of the reference frame and thereference block.
 18. The method according to claim 17, furthercomprising motion compensating the one of the reference frame and thereference block by a motion compensation element.
 19. The methodaccording to claim 18, wherein the dedicated noise reducing filter is anoise adaptive filter which is adapted to estimate noise by evaluatingthe one of the reference frame and the reference block.
 20. The methodaccording to claim 18, wherein the dedicated noise reducing filter is apixel location adaptive filter that takes into account content of theone of the reference frame and the reference block.
 21. The methodaccording to claim 20, wherein an amount of said filtering is dependenton local signal properties.
 22. The method according to claim 21,wherein said filtering of the one of the reference frame and thereference block is made within a prediction loop without directlyaffecting the prediction error which is to be encoded by the encoder.23. The method according to claim 22, wherein said filtering is appliedon each color channel of the one of the reference frame and thereference block separately or in combination.
 24. The method accordingto claim 23, wherein the current block and the reference block are bothin a single frame.
 25. The method according to claim 24, wherein thededicated noise reducing filter is arranged in the encoder and a decoderto reduce or eliminate noise in the reference frame.
 26. The methodaccording to claim 25, wherein filter parameters of the dedicated noisereducing filter of the encoder are included in a coded bitstream. 27.The method according to claim 21, further comprising: estimating filterparameters of the dedicated noise reducing filter of the encoder by adecoder; and providing the filter parameters from the decoder to thededicated noise reducing filter.
 28. The method according to claim 27,wherein said generating of the prediction error includes comparing thecurrent frame to be coded with only a noise-filtered signal part of theone of the reference frame or the reference block.
 29. The methodaccording to claim 28, further comprising bypassing a quantizationelement located in a path between a comparator performing said comparingand an output outputting a coded bitstream.
 30. The method according toclaim 29, wherein said filtering is adaptive with respect to aquantization parameter of the quantization element such that saidfiltering is lower as the quantization parameter increases and saidfilter is greater as the quantization parameter decreases.
 31. Anencoder for reducing noise for coding of noisy images or imagesequences, comprising: first means for adding a prediction error, usedby an encoder to produce one of an encoded frame and an encoded block,to one of a reference frame and a reference block that includes a usefulsignal part and a noise signal part; second means for filtering the oneof the reference frame and the reference block only with a dedicatednoise reducing filter to one of reduce and eliminate the noise signalpart of the one of the reference frame and the reference block; andthird means for generating the prediction error from one of a currentframe and a current block, and at least part of a current or previousone of the reference frame and the reference block.
 32. A decoder forreducing noise for coding of noisy images or image sequences,comprising: first means for adding a prediction error, generated by anencoder from one of an original frame and an original block, to one of areference frame and a reference block that includes a useful signal partand a noise signal part to generate one of a decoded frame and a decodedblock; and second means for filtering the one of the decoded frame andthe decoded block only with a dedicated noise reducing filter to producethe one of the reference frame and the reference block with the noisesignal part thereof reduced.
 33. A method for reducing noise fordecoding of noisy images or image sequences, comprising: adding aprediction error, generated by an encoder from one of an original frameand an original block, to one of a reference frame and a reference blockthat includes a useful signal part and a noise signal part to generateone of a decoded frame and a decoded block; and filtering the one of thedecoded frame and the decoded block only with a dedicated noise reducingfilter to produce the one of the reference frame and the reference blockwith the noise signal part thereof reduced.